Stereo camera systems are used to acquire three-dimensional information about objects. Stereo camera systems are used, for example, in computer vision systems, 3-dimensional tracking applications, object dimensioning applications, object tracking applications, and so on. Typical stereo camera systems include at least two electronic cameras which are mounted at spaced apart locations. The electronic cameras have overlapping fields of view. A computer connected to receive images from each of the cameras can compare the images to derive three-dimensional information about objects in the field of view. Information such as the distances to the objects and the sizes, dimensions and orientations of the objects can be determined by triangulation.
A stereo camera system is typically calibrated by placing a known object in the field of view of the cameras. Computer software which receives images from the cameras can determine the relationship between the cameras from the images of the known object and also compensate for distortions introduced by the lenses. After the stereo camera system has been calibrated then the computer can be used to obtain information about objects whose positions or configurations are not known.
Currently available stereo camera systems use small cameras which have arrays of light sensing elements such as charge coupled devices (“CCDs”), CMOS sensors or the like. A typical camera, as is used in a typical stereo camera system, is mounted to a suitable support in a desired position relative to other camera in the stereo camera system.
A problem with such existing stereo camera systems is that the calibration of the systems can degrade over time. For a stereo camera system to remain perfectly calibrated the fields of view of the cameras must not move relative to one another. In a conventional stereo camera system there is potential for movement in the mounting between the lens holders and the image arrays and there is potential for movement of the support frame itself. It is highly likely that vibration, shocks or the like will cause the field of view of one or more cameras in a stereo camera system to shift over time. If this happens then calibration will be lost.
It has been determined that the performance of a two-camera stereo system can become substantially degraded if any of the cameras moves by more than about ½ pixel relative to the other camera. At currently available camera resolutions each camera should preferably not move relative to the other camera by more than about 0.001 centimeters from the position it had when it was calibrated.
Typically, two lens systems cast images onto two image sensors, configured to run as video cameras. Typically, in such an arrangement, manufacturing tolerances will cause misalignment between each lens and its sensor, and also between one sensor and the other. Moreover, if the lens/sensor combinations have zoom, autofocus or toe-in capability it is likely that the misalignment will change over time.
As will be explained, the present invention can tolerate calibration changes over a period of time and can adapt to mechanical and optical changes over time. The present invention provides a self calibrating stereo camera system that may be performed any time during the life time of the stereo camera system.